Digital image classifier
Digital image classifier
Digital images
A digital image is an image made up of elements called pixels that form a two-dimensional matrix. Each pixel can take a finite value within a range of discrete values. In the most common representation, 8 bits per pixel are used, meaning that each pixel can take up to \(2^8=256\) different values. The resolution of the image is determined by the number of pixels used and the amount of information displayed. In the case of a black and white image, the values between 0 and 255 represent different colour intensities, with 0 being white (or black, depending on the representation) and 255 the opposite, and the intermediate values representing different shades of grey. Colour images have 3 channels, each one being an intensity of red, green and blue (RGB). The combination of the intensities of these 3 primary colours is what generates a colour image. In this way, a black and white image has dimensions height \(\times\) width or \(1\times H\times W\), while a colour image has dimensions \(3\times H \times W\).
MNIST dataset
The MNIST dataset (Modified National Institute of Science and Technology) is a collection of data containing digital images of handwritten digits from 0 to 9. The images are black and white and have dimensions of 28x28 pixels. It is one of the most widely used datasets for learning to develop and use image classifiers, and it has been used as a benchmark to evaluate the performance of artificial intelligence algorithms in the past. In the first part of today’s practical we will use the MNIST dataset.
The original digital-image illustration is not included in this repository.
Important. It is advisable to use the notebook from the previous practical (1-neural-networks.ipynb) to help you solve some of the exercises.
import os
import torch
import torch.nn as nn
from torch.nn import functional as F
import matplotlib.pyplot as plt
import torchvision
from torchvision import datasets, transforms
from torch.utils.data import DataLoadercurrent_dir = os.getcwd()
mnist_dir = os.path.join(current_dir, 'data', 'mnist')
def transform_labels(label):
label_vector = torch.zeros(10)
label_vector[label] = 1
return label_vector
train_dataset = datasets.MNIST(mnist_dir, download=True, train=True, transform=transforms.Compose([
transforms.ToTensor(), # first, convert image to PyTorch tensor
transforms.Normalize((0.1307,), (0.3081,)) # normalize inputs
]), target_transform=transform_labels)
test_dataset = datasets.MNIST(mnist_dir, download=True, train=False, transform=transforms.Compose([
transforms.ToTensor(), # first, convert image to PyTorch tensor
transforms.Normalize((0.1307,), (0.3081,)) # normalize inputs
]), target_transform=transform_labels)The first time you run the code in the previous cell, the MNIST dataset will be downloaded from the PyTorch dataset repository and saved in the specified directory (.data//mnist). You need an internet connection for this step. Once you have downloaded it, you can load it using the path where you saved it without having to download it again.
Let’s look at the information contained in the train dataset and the test dataset that we have loaded:
train_datasettest_datasetWe can see that we are using the mean \(\mu=0.1307\) and standard deviation \(\sigma=0.3081\) values to standardize the images. These values are taken from the PyTorch example on how to use Datasets and Dataloaders (https://docs.pytorch.org/tutorials/beginner/basics/data_tutorial.html). If we look at the dimensions of our datasets, we can see that we have 60000 images of 28x28 values in the train set and 10000 images of 28x28 in the test set.
train_dataset.data.shapetest_dataset.data.shapeEach element of the dataset contains an image and a label:
image, label = train_dataset[42]print(type(image))
print(image.shape)
print(label)As we can see, the label is a vector with 10 values, where the position of the value 1 tells us which digit it is (0-9). In this case we know that it corresponds to the digit 7. The dimensions [1, 28, 28] tell us that the image has a height and width of 28 pixels and a single channel.
Exercise 1. Print the digit of the first element of the train set and of the last element of the test set using the label and the torch.argmax function (use the .item() method at the end if you want to obtain just the integer rather than a tensor).
# write your code hereExercise 2. Display the first image of the train set and the last image of the test set using the plt.imshow function with the argument cmap='gray'. Use torch.squeeze() so that you end up with a 28x28 image instead of 1x28x28 and plt.imshow does not throw an error.
# write your code hereExercise 3. Define a DataLoader for the train set and another one for the test set. For the train set use a batch size of 512 with shuffle=True. For the test set use the same batch size with shuffle=False.
train_loader = None # modify this line
test_loader = None # modify this lineFully Connected Neural Networks
Now the time has come to define our neural network. We will build a model with the following architecture:
- A first linear layer with 784 input features or
in_featuresand 128 hidden neurons orout_features, followed by a ReLU activation function (rectified linear unit, https://docs.pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU). - A second linear layer with 128
in_featuresand 64out_features, followed by a ReLU activation function. - A third and final linear layer with 64
in_featuresand 10out_features, followed by aLogSoftmaxactivation function.
Exercise 4 (written). Why does the input layer of the model have to receive 784 input features? Why does the last layer of the model have to have 10 output values?
Write your answer here
Exercise 5. Modify the constructor and the forward function of the FullyConnected class to define the model described above.
class FullyConnected(nn.Module):
def __init__(self) -> None:
super(FullyConnected, self).__init__()
self.input_layer = nn.Linear(784, 128)
# write your code here
self.softmax = nn.LogSoftmax(dim=1)
def forward(self, x):
x = self.input_layer(x)
x = F.relu(x)
# write your code here
x = self.softmax(x)
return xThe following lines of code are used to load the model parameters onto the graphics card (GPU) in case we have one available. We will also use the device variable in the training loop to load each batch of data onto the GPU. The GPU can parallelize operations such as matrix multiplication, which makes it possible to significantly speed up training and inference with neural networks.
# here we specify that we want to use the GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = FullyConnected().to(device)Now we will define the loss function and the optimizer (the gradient descent algorithm that we use to train the model).
As a loss function we will use cross entropy. Cross entropy is a measure of the similarity between two probability distributions; the more the distributions resemble each other, the smaller the cross entropy between them. This gives us a metric which, when optimized, forces the model to adjust the weights so that, given an input image, the output vector of the model is as similar as possible to the label vector corresponding to that image.
The optimizer we will use is Adam (Adaptive Momentum, https://docs.pytorch.org/docs/stable/generated/torch.optim.Adam.html) with a learning rate of \(10^{-4}\) and a weight decay of \(5·10^{-4}\). The weight decay is a regularization term added to the loss function that penalizes weights which would otherwise become very large during training and lead to overfitting. You can find a more detailed explanation of weight decay (L2 regularization) at https://d2l.ai/chapter_linear-regression/weight-decay.html.
Exercise 6. Define the optimizer (Adam, lr=1e-4, weight_decay=5e-4) and the loss function (CrossEntropyLoss).
optimizer = None # modify this line
loss_function = None # modify this lineFinally, we will train the model.
Exercise 7. Run the following cell and train the model for 10 epochs.
from copy import deepcopy
num_epochs = 10
train_loss_per_epoch, test_loss_per_epoch = [], []
train_accuracy_per_epoch, test_accuracy_per_epoch = [], []
best_model = deepcopy(model)
best_test_loss = torch.inf
for epoch in range(num_epochs):
loss_epoch_train, loss_epoch_test = 0, 0
accuracy_epoch_train, accuracy_epoch_test = 0.0, 0.0
# train in batches
for i, (img, labels) in enumerate(train_loader):
input_data = img.to(device).reshape(-1,784)
labels = labels.to(device)
# Zero your gradients for every batch!
optimizer.zero_grad()
# Make predictions for this batch
outputs = model(input_data)
# Compute the loss and its gradients
loss = loss_function(outputs, labels)
loss.backward()
# Adjust learning weights
optimizer.step()
# Add loss values for this epoch
loss_epoch_train += loss.item() * len(labels)
# compare predictions to labels
total_correct = torch.sum(torch.argmax(outputs, 1) == torch.argmax(labels, 1))
accuracy_epoch_train += total_correct
loss_epoch_train /= len(train_dataset)
accuracy_epoch_train = 100 * (accuracy_epoch_train / len(train_dataset))
train_loss_per_epoch.append(loss_epoch_train)
train_accuracy_per_epoch.append(accuracy_epoch_train)
# inference with test set and evaluate metrics
for j, (img, labels) in enumerate(test_loader):
input_data = img.to(device).reshape(-1,784)
labels = labels.to(device)
outputs = model(input_data)
loss = loss_function(outputs, labels)
loss_epoch_test += loss.item() * len(labels)
total_correct = torch.sum(torch.argmax(outputs, 1) == torch.argmax(labels, 1))
accuracy_epoch_test += total_correct
loss_epoch_test /= len(test_dataset)
accuracy_epoch_test = 100 * (accuracy_epoch_test / len(test_dataset))
test_loss_per_epoch.append(loss_epoch_test)
test_accuracy_per_epoch.append(accuracy_epoch_test)
if loss_epoch_test < best_test_loss:
best_test_loss = loss_epoch_test
best_model = deepcopy(model)
print('epoch: {}, train loss: {:.3f}, train accuracy: {:.3f}'.format(epoch+1, loss_epoch_train, accuracy_epoch_train))
print('epoch: {}, test loss: {:.3f}, test accuracy: {:.3f}\n'.format(epoch+1, loss_epoch_test, accuracy_epoch_test))Exercise 8. Use matplotlib to display two plots: - One showing the loss per epoch curve for the train set and another curve with the loss per epoch for the test set. Use the colour blue ('b') for the train curve and the colour red ('r') for the test curve. Label the x axis “Epochs”, the y axis “Cross Entropy Loss” and show the legend. Use a fontsize of 14 for the axes and 12 for the legend. - One curve showing the accuracy per epoch for the train set and another curve with the accuracy per epoch for the test set, with the same characteristics as the previous plot. Label the y axis “Accuracy”.
epochs = range(1, num_epochs+1)
fig, ax = plt.subplots(1,2, figsize=(12, 5))
# write your code hereExercise 9. Make a prediction with the trained model using a random element from the test set. Use torch.randint to generate a random number between 0 and 10000, select that element from the test set and run inference with the trained model. You will have to convert the image into a vector of dimensions (1, 784) in order to be able to make the prediction. From the output vector, take the position of the largest value as your model’s prediction. Then, print the prediction and display the selected element with plt.imshow. Is your prediction correct? (note: an incorrect prediction will not deduct any points, it is normal for the accuracy not to be 100%)
random_img = torch.randint(len(test_dataset), (1,)).item()
# write your code hereExercise 10. Modify the model you defined in exercise 5 and customize it however you like. You can experiment with the number of layers, the number of neurons in each layer, the activation function (sigmoid, tanh, relu, leaky relu, etc. (https://machinelearningmastery.com/activation-functions-in-pytorch/, https://docs.pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity), dropout, etc. You can also try different hyperparameters such as the batch size, learning rate, weight decay or number of epochs. In order for the model to work with the training loop code that we used earlier, the input layer of the neural network must have 784 neurons. Are you able to get the model to reach an accuracy of more than 97% on the test set?
# write your code hereExercise 11. Print the number of parameters of the final model you used.
# write your code hereConvolutional Neural Networks
Follow the instructions of the PyTorch tutorial on how to train a classifier using Convolutional Neural Networks: https://docs.pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html
Copy the code blocks from the tutorial into this notebook (except the first 2, which are already copied) and run them.
Note: When you load the model you have saved, in the lines
net = Net()
net.load_state_dict(torch.load(PATH, weights_only=True))remove the argument weights_only=True, so that you end up with
net.load_state_dict(torch.load(PATH))import os
import torch
import torchvision
import torchvision.transforms as transformscurrent_dir = os.getcwd()
cifar_dir = os.path.join(current_dir, 'data', 'CIFAR10')
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
batch_size = 4
trainset = torchvision.datasets.CIFAR10(root=cifar_dir, train=True,
download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
shuffle=True, num_workers=2)
testset = torchvision.datasets.CIFAR10(root=cifar_dir, train=False,
download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
shuffle=False, num_workers=2)
classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')Exercise 12. Modify the convolutional neural network (constructor and forward method) so that it has the following structure:
- A first convolutional layer (nn.Conv2d) with 3
in_channels, 8out_channelsand akernel_sizeof 3 - A second convolutional layer with 8
in_channels, 16out_channelsand akernel_sizeof 3 - A first max pooling operation (nn.MaxPool2d) with a
kernel_sizeof 2 and astrideof 2 - A third convolutional layer with 16
in_channels, 16out_channelsand akernel_sizeof 3 - A fourth convolutional layer with 16
in_channels, 32out_channelsand akernel_sizeof 3 - A second max pooling operation with a
kernel_sizeof 2 and astrideof 2 - A first linear layer (nn.Linear) with 800
in_featuresand 128out_features - A second linear layer (nn.Linear) with 128
in_featuresand 10out_features - A LogSoftmax activation on the last layer
F.log_softmax(x, dim=1)
Each convolutional layer and the first linear layer must be followed by a ReLU activation function F.relu(x). The second linear layer (and last layer of the neural network) must be followed by the LogSoftmax activation (F.log_softmax).
In the forward method, after the second max_pooling operation you must use torch.flatten to convert each element of your batch into a vector that you can use as input for the linear layers. The output must be a tensor of dimensions (batch size, 10) that you can compare with the label of the same dimensions. The 10 output neurons correspond to the 10 classes of the dataset.
class ConvNet(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(3, 8, kernel_size=3)
self.conv2 = nn.Conv2d(8, 16, kernel_size=3)
self.max_pool = nn.MaxPool2d(2, 2)
# write your code here
self.output = nn.Linear(128, 10)
def forward(self, x):
x = F.relu(self.conv1(x))
# write your code here
x = self.max_pool(x)
x = torch.flatten(x, 1) # flatten all dimensions except batch
x = F.log_softmax(self.output(x), dim=1)
return x
net = ConvNet()Exercise 13. Print the number of parameters of the convolutional network. Does it have more or fewer parameters than the fully connected model you created earlier?
# write your code hereExercise 14. Train the model for 10 epochs using the training loop provided by the PyTorch tutorial. Use the GPU if you have one available with
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
print(device)
net.to(device)# modify whichever lines of code are necessary in order to use the GPU if applicable
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
for epoch in range(10): # loop over the dataset multiple times
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
# get the inputs; data is a list of [inputs, labels]
inputs, labels = data
# zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# print statistics
running_loss += loss.item()
if i % 2000 == 1999: # print every 2000 mini-batches
print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
running_loss = 0.0
print('Finished Training')Exercise 15. Print the accuracy of your model on the test set.
# write your code hereExercise 16. Finally, make a prediction taking a random element from the test set. Display the image you have selected and print the name of the predicted class and the name of the correct class. Did your model get it right? What probability did the model assign to the correct class of the example you selected?
# write your code hereExercise 17 (additional exercise, 2 extra points). Modify the neural network you created in exercise 12 and customize it however you like. Are you able to obtain more than 80% accuracy on the test set?